Delivery of video content is available through a variety of hardware platforms, such as mobile devices or living room-connected devices (e.g., gaming consoles, streaming devices, and smart televisions), laptop computers, etc. Different hardware platforms may have different media players, which may each be implemented differently. For example, the media players may have been developed internally by a media streaming service or may be pre-installed as part of the hardware platform. The different implementations may react to different streaming conditions differently. The media streaming service thus needs to test these different media player implementations to determine whether playback problems may occur.
The testing of the different media player implementations may be performed manually. For example, a tester may physically watch a video being played by the different platforms. However, testing the platforms in this way is very labor intensive.